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T— I Abstract. Multiview geometry is the study of two-dimensional images 

T~H of three-dimensional scenes, a foundational subject in computer vision. 

^^ We determine a universal Grobner basis for the multiview ideal of n 

v^ generic cameras. As the cameras move, the multiview varieties vary in a 

' ~~* family of dimension lln— 15. This family is the distinguished component 

I ^ of a multigraded Hilbert scheme with a unique Borel-fixed point. We 

present a combinatorial study of ideals lying on that Hilbert scheme. 



O 1. Introduction 

. Computer vision is based on mathematical foundations known as multi- 

« view geometry [71|9] or epipolar geometry [11, §9]. In that subject one studies 

C^ the space of pictures of three-dimensional objects seen from n > 2 cameras. 

^ Each camera is represented by a 3 x 4- matrix Ai of rank 3. The matrix 

'""' specifies a linear projection from P^ to P^, which is well-defined on P^\{/j}, 

y—{ where the focal point fi is represented by a generator of the kernel of Ai . 

!>■ The space of pictures from the n cameras is the image of the rational map 

r- (1) (t>A : IP' -^ (IP'r, X ^ (Aix,^2X,...,A„x). 

OO 

p^ The closure of this image is an algebraic variety, denoted Va and called the 

[^ multiview variety ol the given n-tuple of 3 x 4- matrices A = {Ai,A2, . . . , A^)- 

(^ In geometric language, the multiview variety Va is the blow-up of P^ at the 

^~~^ cameras /i, . . . , /„, and we here study this threefold as a subvariety of (P^)". 

. . The multiview ideal J a is the prime ideal of all polynomials that vanish 

^ on the multiview variety Va- It lives in a polynomial ring K[x,y,z\ in 3n 

K> unknowns {xi,yi,Zi), i = 1,2, . . . ,n, that serve as coordinates on (P^)". In 

^ Section 2 we give a determinantal representation of Ja for generic A, and 

identify a universal Grobner basis consisting of multilinear polynomials of 

degree 2, 3 and 4. This extends previous results of Heyden and Astrom [12]. 

The multiview ideal Ja has a distinguished initial monomial ideal Mn that 

is independent of A, provided the configuration A is generic. Section 3 gives 

an explicit description of M„ and shows that it is the unique Borel-fixed 

ideal with its Z"-graded Hilbert function. Following [3], we introduce the 

multigraded Hilbert scheme ?^„ which parametrizes Z^-homogeneous ideals 

in K[x,y,z] with the same Hilbert function as M„. We show in Section 6 

that, for n > 3, Tin has a distinguished component of dimension lln — 15 

which compactifies the space of camera positions studied in computer vision. 

For two cameras, that space is an irreducible cubic hypersurface in 7^2 — ^^■ 
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Figure 1 . A multiview variety Va for n = 3 cameras degen- 



erates into six copies of ^ 



and one copy of ^ 



Section 4 concerns the case when n < 4 and the focal points fi are among 
the coordinate points (1:0:0:0), . . . , (0:0:0:1). Here the multiview variety Va 
is a toric threefold, and its degenerations are parametrized by a certain toric 
Hilbert scheme inside Tin- Each initial monomial ideal of the toric ideal J a 
corresponds to a three-dimensional mixed subdivision as seen in Figure [l} A 
classification of such mixed subdivisions for n = 4 is given in Theorem |4.3[ 

In Section 5 we place our n cameras on a line in P^. Moving them very 
close to each other on that line induces a two-step degeneration of the form 



trinomial ideal 



binomial ideal 



monomial ideal. 



(2) 

We present an in-depth combinatorial study of this curve of multiview ideals. 

In Section 6 we finally define the Hilbert scheme Tin, and we construct the 

space of camera positions as a GIT quotient of a Grassmannian. Our main 



result (Theorem 6.3) states that the latter is an irreducible component of Tin- 
As a key step in the proof, the tangent space of Tin at the monomial ideal in 
([2]) is computed and shown to have the correct dimension lln— 15. Thus, the 
curve ([2]) consists of smooth points on the distinguished component of Tin- 
For n > 3, our Hilbert scheme has multiple components. This is seen from 
our classification of monomial ideals on Ti^, which relates closely to [H §5]. 
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2. A UNIVERSAL GrOBNER BASIS 

Let K be any algebraically closed field, n > 2, and consider the map (pA 
defined as in (jTl) by a tuple A = {Ai, A2, . . . , An) of 3 x 4- matrices of rank 3 
with entries in K. The subvariety Va = image(</>A) of (P^)" is the multiview 
variety, and its ideal Ja C K[x,y,z] is the multiview ideal. Note that Ja is 
prime because its variety Va is the image under cpA of an irreducible variety. 

We say that the camera configuration A is generic if all 4 x 4-minors of 
the (4 X 3n)-matrix ^Aj A^ • • • ^^] are non-zero. In particular, if A is 
generic then the focal points of the n cameras are pairwise distinct in P'^. 
For any subset a = {o"i, . . . , ag} ^ [n] we consider the 3s x (s + 4)-matrix 

\Aa, Pa, ••• 



Aa 



Aa2 Pa2 '■• 

U^, ••• pa^ 



where pi := [rcj yi Zi\ for i G [n\. Assuming s > 2, each maximal minor 
of Afj is a homogeneous polynomial of degree s = |(t| that is linear in p, 
for i £ a. Thus for s = 2,3,... these polynomials are bilinear, trilinear, 
etc. The matrix A^ and its maximal minors are considered frequently in 
multiview geometry |1H I12j . Recall that a universal Grobner basis of an 
ideal is a subset that is a Grobner basis of the ideal under all term orders. 
The following is the main result in this section. 

Theorem 2.1. If A is generic then the maximal minors of the matrices A^ 
for 2 < \a\ < 4 form a universal Grobner basis of the multiview ideal Ja- 

The proof rests on a sequence of lemmas. Here is the most basic one. 

Lemma 2.2. The maximal minors of A^ for |(t|>2 lie in the prime ideal Ja- 

Proof: If [pi, - - . ,pn) G (i^^)" represents a point in image((/)A) then there 
exists a non-zero vector q £ K"^ and non-zero scalars ci, . . . ,Cn E K such 
that AiQ = CiPi for i = 1,2, . . . ,n. This means that the columns of Au are 
linearly dependent. Since A^ has at least as many rows as columns, the 
maximal minors of Au must vanish at every point p gVa- CH 

Later we shall see that when A is generic, J a has only one initial monomial 
ideal up to symmetry. We now identify that ideal. Let M„ denote the ideal 
in K[x,y,z] generated by the Q) quadrics XiXj, the 3(3) cubics Xiyj-yk, and 
the (2) quartics yiyjykVh where i,j,k,l runs over distinct indices in [n]. 

We fix the lexicographic term order -< on K[x,y,z] which is specified by 
xi>~ ■ ■ ■ >~Xn>-yi>- ■ ■ ■ >~yn>~zi>~ ■ ■ ■ >~Zn- Our goal is to prove that the initial 
monomial ideal in^(Jyi) is equal to M„. We begin with the easier inclusion. 
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Lemma 2.3. If A is generic then Mn ^ in^(Jyi). 

Proof: The generators of Mn are the quadrics XiXj, the cubics Xiyji/k, and 

the quartics ynijykyi- By Lemma 2.2, it suffices to show that these are the 

initial monomials of maximal minors oi Aujx, Aujk} and Auji^n respectively. 

For the quadrics this is easy. The matrix Aujx is square and we have 

Al 



(3) det(^|,,}) 



det 



Af 
A^ 

.^3 



Xi 

Vi 

Zi 







0" 











\An 




Xj 


= det 


4 


Vj 




^L 







XiXj + lex. lower terms. 



(4) 



A 



{ijk} 



where A^ is the rth row of At. The coefficient of XiXj is non-zero because 
A was assumed to be generic. For the cubics, we consider the 9 x 7-matrix 

O" 


Vk_ 

Now, Xiyjyk is the lexicographic initial monomial of the 7 x 7-determinant 
formed by removing the fourth and seventh rows of Aujk} ■ Here we are using 
that, by genericity, the vectors A^ , Af , A^ , Af, are linearly independent. 
Finally, for the quartic monomial yiyjykVi we consider the 12 x 



Ai 


Pi 





A, 





Pi 


Ak 









matrix 



(5) 



A 



{ijki} 



Ai 


Pi 








A, 





Pi 





Ak 








Pk 


Ai 
















Pi 



Removing the first row from each of the four blocks, we obtain an 8 x 8- 
matrix whose determinant has yiyjykVi as its lex. initial monomial. D 



We extend each camera matrix Aj to an invertible 4 x 4-matrix B^ 



The next step towards our proof of Theorem 2.1 is to express the mul- 
tiview variety V^ as a projection of a diagonal embedding of P^. This will 
put us in a position to utilize the results of Cartwright and Sturmfels in [3] . 

[A, 
by adding a row bi at the top. Our diagonal embedding of P^ is the map 

(6) Vb : P^ ^ (P^)", X ^ (5ix, 52X, . . 

Let V^ := image^tpB) C (P^)" and J^ C K[w,x,y,z 
Here {wi : Xi : yi : Zi) are coordinates on the rth copy of I 
are coordinates on (P^)". The ideal J is generated by the 2 x 2-minors of 



, S„x). 



its prime ideal. 
^ and (u),x, y, z) 



(7) 





Wl 




W2 


B-,' 


Xl 

yi 


B,' 


X2 
2/2 




_Zl_ 




_Z2_ 



B 



-1 



Wn 
Xn 

yn 

Zn 
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This is a 4 X n-matrix. Now consider the coordinate projection 

vr : (p3)" -^ (p2)" , (wi -.Xi-.yi: Zi) ^ {xi : yi : Zi) for i = 1, ... , n. 

The composition vr o ■i/)^ is a rational map, and it coincides with (p^ on its 
domain of definition P^\{/i, . . . , /„}. Therefore, Va = '7t{V^) and 

(8) Ja = J^nK[x,y,z]. 

The polynomial ring K[w, x, y, z] admits the natural Z^-grading deg(ti;i) = 
deg(xi) = deg(yi) = deg(zj) = ei where e, is the standard unit vector in M". 
Under this grading, K[w,x,y, z]/J^ has the multigraded Hilbert function 

The multigraded Hilbert scheme i/4,„ which parametrizes Z"-homogeneous 
ideals in K[w,x,y,z] with that Hilbert function was studied in ^. More 
generally, the multigraded Hilbert scheme Hfin represents degenerations of 
the diagonal P"~^ in (^p^-i-jn. ^^^ ^^^y. ^ ^^^ ^_ p^^. ^^^ general definition 
of multigraded Hilbert schemes see [TO]. It was shown in [3j that H^^n has 
a unique Borel- fixed ideal Z^^n- Here Borel- fixed means that Z^^n is stable 
under the action of i3" where B is the group of lower triangular matrices in 
PGL(d, K). Here is what we shall need about the monomial ideal Z^^n- 

Lemma 2.4. (Cartwright-Sturmfels [3l §2] and Conca [H §5]) 

(1) The unique Borel- fixed monomial ideal Z4^„ on H^^n is generated by 
the following monomials where i,j,k,l are distinct indices in [n]: 

WiWj, WiXj, Wiyj, XiXj, Xiyjyk, yiVjykVi- 

(2) This ideal Z/^^n is the lexicographic initial ideal of J^ when B is 
sufficiently generic. The lexicographic order here isw>~x>~y>~z 
with each block ordered lexicographically in increasing order of index. 

Using these results, it was deduced in [3] that all ideals on Hi^^n are rad- 
ical and Cohen-Macaulay, and that -ff4,„ is connected. We now use this 



distinguished Borel- fixed ideal Z4^„ to prove the equality in Lemma 2.3 
Lemma 2.5. If A is generic then Mn = in_<(J4). 

Proof: We fix the lexicographic term order -< on K[w,x,y, z] and its re- 

strictior 

Lemma 



strictio n to K\x,y,z]. Lemma 2.4 (1) shows that M„ = ^4^^ n K[x,y, z]. 
(2) states that Z^^n = iii-<(J^) when B is generic. The lexico- 
graphic order has the important property that it allows the operations of 
taking initial ideals and intersections to commute [5l Chapter 3]. Therefore, 

in^ (Ja) = m^{J^nK[x,y,z]) 
= m^{J^)nK[x,y,z] 
= Zi^n^K[x,y,z\ = Mn. 
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This identity is valid whenever the conclusion of Lemma 2.4 (2) is true. We 
claim that, for this to hold, the appropriate genericity notion for B is that all 
4 X 4-minors of the (4 X 4n)-matrix iBf BJ ••• B'^'l are non-zero. Indeed, 
under this hypothesis, the maximal minors of the 4s x (s + 4)-matrix 



5. 



Ba, P. 



o-l 







B^ 



B, 



Pcra 







Pa. 



, where Pi := [ 



Wj X, 



yi Zi\ for i G 



n\ 



have non- vanishing leading coefficients. We see that Z^^n ^ iii^(<^ ) by 



reasoning akin to that in the proof of Lemma 2.3 The equality Z4_„ = 
in^ {J^) is then immediate since Z^^^ is the generic initial ideal of J^ . Hence, 
for any generic camera positions A, we can add a row to Ai and get Bi that 
are "sufficiently generic" for Lemma 2.4 (2). This completes the proof. D 



Proof of Theorem 2.1: Lemma 2.5 and the proof of Lemma 2.3 show that 
the maximal minors of the matrices A^ for 2 < |o"| < 4 are a Grobner basis 
of Ja for the lexicographic term order. Each polynomial in that Grobner 
basis is multilinear, thus the initial monomials remain the same for any 
term order satisfying Xi >- yi >- Zi for i = 1, 2, . . . , n. So, the minors form 
a Grobner basis for that term order. The set of minors is invariant under 
permuting {xj,^^, Zj} for each i. Moreover, the genericity of A implies that 
every monomial which can possibly appear in the support of a minor does 
so. Hence, these minors form a universal Grobner basis of J^^. D 

Remark 2.6. Computer vision experts have known for a long time that 
multiview varieties Va are defined set-theoretically by the above multilinear 
constraints of degree at most 4. We refer to work of Heyden and Astrom 
|12^ [T3] . What is new here is that these constraints define Va in the strongest 
possible sense: they form a universal Grobner basis for the prime ideal Ja- 

The n cameras are in linearly general position if no four focal points 
are coplanar and no three are collinear. While the number of multilinear 
polynomials in our lex Grobner basis of J a is (2) +3(3) + (4) , far fewer 
suffice to generate the ideal Ja when A is in linearly general position. 

Corollary 2.7. If A is in linearly general position then the ideal J a is 
minimally generated by 



bilinear and (3) trilinear polynomials. 



Proof: This can be shown for n < 4 by a direct calculation. Alternatively, 
these small cases are covered by transforming to the toric ideals in Section 
4. First map the focal points of the cameras to the torus fixed focal points of 
the toric case, followed by multiplying each Ai by a suitable gi £ PGL(3, K). 
Now let n > 5. For any three cameras i, j, k, the maximal minors of Q 
are generated by only one such maximal minor modulo the three bilinear 
polynomials ([3]). Likewise, for any four cameras i, j, k and /, the maxi- 
mal minors of (pi) are generated by the trilinear and bilinear polynomials. 
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This implies that the resulting (2) + (3) polynomials generate J a, and, by 
restricting to two or three cameras, we see that they minimally generate. D 

3. The Generic Initial Ideal 
We now focus on combinatorial properties of our special monomial ideal 

M„ = {xiXj, Xiyjyk, ViVjykyi ■ V ij,k,l £ [n] distinct). 

We refer to M„ as the generic initial ideal in multiview geometry because it 
is the lex initial ideal of any multiview ideal Ja after a generic coordinate 
change via the group G" where G = PGL(3,-ff). Indeed, consider any rank 
3 matrices Ai,A2, ■ ■ ■ ,An G K^^'^ with pairwise distinct kernels K{fi}. If 
9 = (9i;52) • • • )5n) is generic in G" then g o A is generic in the sense that 
all 4 X 4-minors of the matrix [((71^1)^ (52^2)"^ ■ ■ ■ (ffn^n)^] are non-zero. 
Thus, by the results of Section 2, M„ is the initial ideal of JgoA, or, using 
standard commutative algebra lingo, M„ is the generic initial ideal of Ja- 

Since M„ is a squarefree monomial ideal, it is radical. Hence M„ is the 
intersection of its minimal primes, which are generated by subsets of the 
variables Xi and yj. We begin by computing this prime decomposition. 

Proposition 3.1. The generic initial ideal Mn is the irredundant intersec- 
tion of (0) + 2(2) monomial primes. These are the monomial primes Pijk 
and Qij C K[x, y, z] defined below for any distinct indices i,j, k £ [n]: 

• Pijk is generated by xi, . . . ,Xn and all yi with I {i, j, k}, 

• Qij is generated by all xi for I ^ i and yi for I {i,j}- 

Proof: Let L denote the intersection of all P^jk and Qij. Each monomial 
generator of M„ lies in Pijf. and in Qij, so M„ C L. For the reverse inclusion, 
we win show that y(M„) is contained in V{L) = {UV{Pijk)) U {\JV{Qij)). 

Let {x,y, z) be any point in the variety V{Mn). First suppose Xi = for 
all i € [n]. Since yiyji/kyi = for distinct indices, there are at most three 
indices i,j, k such that y^, yj and yj- are nonzero. Hence (5, y, z) E V{Pijk). 

Next suppose Xi 7^ 0. The index i is unique because XiXj G M„ for all 
j ^ i. Since Xiyjij^ = for all j, k ^ i, we have yj / for at most one index 
j 7^ i. These properties imply (x,y,z) G V{Qij). D 

We regard the monomial variety V{Mn) as a threefold inside the product 
of projective planes (P^)". If the focal points are distinct, Va has a Grobner 
degeneration to the reducible threefold V{Mn). The irreducible components 
of V{Mn) are 

(9) V{Pijk) ~ P^ X pi X pi and V{Qij) ~ P^ x P^ 

We find it convenient to regard (P^)" as a toric variety, so as to identify it 
with its polytope (A2)"', a direct product of triangles. The components in 
(jol) are 3-dimensional boundary strata of (P^)", and we identify them with 
faces of (A2)". The corresponding 3-dimensional polytopes are the 3-cube 
and the triangular prism. The following three examples illustrate this view. 
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-J3. 




Figure 2. The variety of the generic initial ideal M2 seen 
as two adjacent facets of the 4-dimensional polytope A2 x A2. 





Figure 3. The monomial variety ^(Ms) as a subcomplex of (A2)" 



Example 3.2. [Two cameras [n = 2)] The variety of M2 = (xi) n (X2) is a 
hypersurface in P^ x P^ . The two components are triangular prisms P^ x P^ , 



which are glued along a common square 



^^, as shown in Figure^ D 



Example 3.3. [Three cameras (n = 3)] The variety of M3 is a threefold 
in P^ X P^ X P^. Its seven components are given by the prime decomposition 

M3 = (xi,x2,yi) n (xi,x2,y2) n {xi,x-i,yi) 

n {xi,x^,y^) n (X2,a;3,y2) n {x2,x-i,y-i) n (xi,X2,X3). 

The last component is a cube P"*^ x P-*^ x P"*^, and the other six components 
are triangular prisms P^ x P^. These are glued in pairs along three of the 
six faces of the cube. For instance, the two triangular prisms V{xi,X2-,yi) 
and y(xi, X3, yi) intersect the cube y(xi, X2, X3) in the common square face 
y(xi,X2,X3,yi) ~ P^ X P^. This polyhedral complex lives in the boundary 
of (A2)'^, and it shown in Figure pi Compare this picture with Figure 111 D 
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Example 3.4. [Four cameras (n = 4)] The variety V^M^) is a threefold 
in (P'^)'^, regarded as a 3-dimensional subcomplex in the boundary of the 8- 
dimensional polytope (A2)^. It consists of four cubes and twelve triangular 
prisms. The cubes share a common vertex, any two cubes intersect in a 
square, and each of the six squares is adjacent to two triangular prisms. D 



From the prime decomposition in Proposition 3.1 we can read off the 
multidegree [171 §8.5] of the ideal M„. Here and in what follows, we use the 
natural Z"-grading on K[x,y,z] given by deg{xi) = deg(yi) = deg(zi) = Cj. 
Each multiview ideal J^ is homogeneous with respect to this Z"-grading. 

Corollary 3.5. The multidegree of the generic initial ideal Mn is equal to 

(10) C{K[x,y,z]/Mn;t)) = t?t^••t^( Y. ^+ E ^ 

\l<i<j<k<n * ^ ^ l<i,j<n * ^ 

A more refined analysis also yields the Hilbert function in the Z"-grading. 
Theorem 3.6. The multigraded Hilbert function of K[x,y,z]/Mn equals 

(11) N" ^ N, (ui,...,u„) ^ ( ^ )-Z^( 3 )• 

Proof: Fix w G N". A ET-basis «B„ for {K[x,y,z]/Mn)u is given by all 
monomials x°"]fz^ M„ such that a^h + c = u. Therefore, either (i) a = 
and at most three components of h are non-zero; or (ii) a 7^ 0, in which case 
only one Oj can be non-zero and hj 7^ for at most one j G M\{*}- 

We shall count the monomials in ^B^. Monomials of type (i) look like 
y z'^, with at most three nonzero entries in h. Also, h determines c since 
Ci = Ui— bi for all i G [n], and so we count the number of possibilities for y . 
There are Ui choices for bi 7^ 0, and thus U := ui + - ■ ■+Un many monomials 
in the set 3^ := {y^^ : 1 < hi < Ui, i = 1, . . . , n}. The factor y^ in y^z'^ is the 
product of 0, 1, 2 or 3 monomials from y with distinct subscripts. 

To resolve over-counting, consider a fixed index i. There are C^^) ways of 
choosing two monomials from y with subscript i and (^J) ways of choosing 
three monomials from y with subscript i. Also, there are C^*) {U — Ui) ways 
of choosing two monomials from y with subscript i and a third monomial 
with a different subscript. Hence, the number of choices for y in y z'^ is 



M 






U \ v-^ / Uj \ v-^ / Ua \ v-^ / U 



3; ^\3j ^^\2l ' ^"'V2 

i—1 ^ ' 2 — 1 ^ '^ i—1 



Y^f "M ttSTI "M , Y^y. 



For case (ii) we count all monomials x°"\fz'^ G '^u with aj 7^ and all other 
Oj = 0. It suffices to count the choices for the factor x"y . For fixed i, there 
are ("2 ) monomials of the form x'^'-y-^ with Oj + 6j < Ui and Oj > 1. Such 
a monomial may be multiplied with y-^ such that j ^ i and < 6j < Uj. 



10 
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This amounts to choosing zero or one monomial from y\{yi, yf,..., y"'} for 
which there are 1 + U — Ui choices. Hence, there are 



i^ + U] 



4 = 1 



Ui + 1 

2 



E 

i=l 



Ui + 1 



monomials in 55^ of type (ii). Adding the two expressions, we get 



IS. 



1 + U + 



= 1 + U 



n 

+ {i + u)u-Y, 



i=l 

Ui + 2 



E 

i=l 



C/ + 3 
3 



E 



u^ + 2 
3 



D 



Our analysis of M„ has the following implication for the multiview ideals 
J A. Note that these are Z"-homogeneous for any camera configuration A. 



; An) 



with 



Theorem 3.7. For an n-tuple of camera matrices A = (yli, 

rank(^j) = 3 for each i, the multiview ideal J a has the Hilbert function (11) 

if and only if the focal points of the n cameras are pairwise distinct. 

Proof: The if-direction follows from the argument in the first paragraph of 
this section. If the n camera positions fi = ker(^j) are distinct in P^ then 
Mn is the generic initial ideal of J a, and hence both ideals have the same 
Z"-graded Hilbert function. For the only-if-direction we shall use: 

(12) If Q G PGL(4,K) and AQ := {AiQ, . . .,AnQ), then Ja = Jaq- 

This holds because Q defines an isomorphism on P^ and hence 4>a as in (II]) 
has the same image in (P^)" as ipAQ- 

Suppose first that n = 2 and Ai and A2 have the same focal point and 
hence the same (three-dimensional) rowspace W . We can map W to the 
hyperplane {xi = 0} by some Q G PGL(4, i^), and (12) ensures that J a = 
Jaq ■ Thus we may assume that ^i = [ Ci ] and A2 = [ C2 ] where 
Ci and C2 are invertible matrices and is a column of zeros. Choosing 
/i = /2 = (1, 0, 0, 0) as the top row of Bi and B2 (as in Section 2), we have 



B-^' 



1 

Cf^ 



B2' 



1 







C2-1 



The ideal J is generated by the 2x2 minors of the matrix ¥In which is 



D 



wi t(;2 

pi(xi,yi,zi) qi{x2,y2,Z2) 

P2{xi,yi,zi) q2{x2,y2,Z2) 

P3ixi,yi,zi) qz{x2,y2,Z2) 
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where the pi's and g^'s are hnear polynomials. The ideal / generated by the 
2x2 minors of the submatrix of D obtained by deleting the top row lies on 
the Hilbert scheme H32 from [3j and hence K[x, y, z]/I has Hilbert function 



N^ 



N, (ni,n2)i-> 



U1+U2 + 2 



For (mi,M2) = (111)) this has value 6. Since I ^ Ja = J^ (^ K[x,y, z], the 



Hilbert function of K[x,y,z]/JA has value < 6, while (11) evaluates to 8. 

If n > 2, we may assume without loss of generality that Ai and A2 
have the same rowspace. The argument for n = 2 shows that Ja = J^ H 
K[x, y, z\ D I. The Hilbert function value of K[x, y, z]/Ja in degree ei + 62 
is again 8, while the Hilbert function value of K[x, y, z]/I in degree ei + 62 
coincides with the value 6 for K[xi,yi, zi,X2, y2, 22]/!- So we again conclude 
that K[x,y, z]/Ja does not have Hilbert function ( |11[ ). D 

For G = PGL(3, K), the product G" acts on K[x, y, z] by left-multiplication 



(51 



,9n) 



Xi 




Xi 


Vi 


= gi 


Vi 


^i 




Zi 



An ideal / in K[x, y, z] is said to be Borel-fixed if it is fixed under the induced 
action of ^B" where B is the subgroup of lower triangular matrices in G. 

Proposition 3.8. The generic initial ideal Mn is the unique ideal in K[x, y, z] 



that is Borel-fixed and has the Hilbert function (11) in the 17^ -grading 



Proof: The proof is analagous to that of O Theorem 2.1], where Zrf„ plays 
the role of M„. The ideal M„ is Borel-fixed because it is a generic initial 
ideal. The same approach as in [6| §15.9.2] can be used to prove this fact. 
The multidegree of any Z"-graded ideal is determined by its Hilbert series 



\n\ Claim 8.54]. Thus any ideal / with Hilbert function ( 11 ) has multidegree 
(10). Let / be such a Borel-fixed ideal. This is a monomial ideal. 



Each maximum-dimensional associated prime P of I has multidegree ei- 
ther tjtl ■ ■ ■ tl/{titjtk) or tft| • • • tl/{tftj), by dZl Theorem 8.53]. In the first 
case P is generated by 2n — 3 indeterminates, one associated with each of 
the three cameras i,j, k and two each from the other n — 3 cameras. Borel- 
fixedness of I tells us that the generators indexed by each camera must be 
the most expensive variables with respect to the order -<. Hence P = Pijt- 
Similarly, P = Qij in the case when P has multidegree t\t\ ■ ■ ■ t\/{tjtj). 

Every prime component of Mn is among the minimal associated primes 
of /. This yields the containments / C v/ C M„. Since / and M„ have the 
same Z"-graded Hilbert function, the equality / = M^ holds. D 

The Stanley -Reisner complex of a squarefree monomial ideal M in a poly- 
nomial ring if [fi, . . . , is] is the simplicial complex on {1, . . . , s} whose facets 
are the sets [s\\ct where P^ := {ti : i £ a} is a minimal prime of M. A 
shelling of a simplicial complex is an ordering Fi, F2, . . . , Fg of its facets 
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such that, for each 1 < j < q, there exists a unique minimal face of Fj (with 
respect to inclusion) among the faces of Fj that are not faces of some earlier 

Definition 2.1]. If the Stanley- Reisner complex of M 

. . . ,ts]/M is Cohen-Macaulay [18, Theorem 2.5]. 



facet Fi, i < j; see 
is shellable, then K[ti 



Proposition 3.9. The Stanley-Reisner complex of the generic initial ideal 
Mn is shellable. Hence the quotient ring K[x,y, z]/Mn is Cohen-Macaulay. 

Proof: This proof is similar to that for Z^^ „ given in {3l Corollary 2.6]. Let 
A„ denote the Stanley-Reisner complex of the ideal M„. By Proposition|3.1[ 



there are two types of minimal primes for M„, namely Pijk and Qjj, which 
we describe uniformly as follows. Let P = (pij) be the 3 x n matrix whose 
ith column is [xi yi zi]^ . For u G {0, 1,2}" define Pu := {pij : i < Uj, 1 < 
j < n). Then the minimal primes Pijk of M„ are precisely the primes Pu 
as u varies over all vectors with three coordinates equal to one and the rest 
equal to two, and the minimal primes Qij are those Pu where u has one 
coordinate equal to zero, one coordinate equal to one and the rest equal 
to two. The facet of A„ corresponding to the minimal prime Pu is then 
Pu '■= {Pij '■ "Uj < i < 3, 1 < J < n}. We claim that the ordering of 
the facets Fu induced by ordering the it's lexicographically starting with 
(0, 1, 2, 2, . . . , 2) and ending with (2, 2, ... , 2, 1, 0) is a shelling of A„. 

Consider the face rju := {pij : j > l,i = Uj + 1 < 2} oi the facet Fu- We 
will prove that r]u is the unique minimal one among the faces of Fu that have 
not appeared in a facet F^' for u' < u. Suppose G is a face of Fu that does 
not contain rju- Pick an element pu^+ij G Vu\G. Then j > 1, Uj < 1 and so 
if Fu is not the first facet in the ordering, then there exists i < j such that 
Ui > because u > (0, 1, 2, 2, ... , 2) and of the form described above. Pick 
i such that i < j and Ui > and consider Fu+e^-e^ = Fu\{puj+i,j} U {pui,i}- 
Then u + Cj — Ci < u and G is a face of Fu+e^-ei ■ Conversely, suppose G is a 
face of Fu that is also a face of Fu' where u' < u. Since ^ n'- = X] ^i) there 
exists some j > I such that u'- > Uj. Therefore, G does not contain pu^+ij 
which belongs to rju- Therefore, rju is not contained in G. D 



4. A ToRic Perspective 

In this section we examine multiview ideals J a that are toric. For an 
introduction to toric ideals we refer the reader to |20] . We now assume that, 
for each camera i, each of the four torus fixed points in P^ either is the 
camera position or is mapped to a torus fixed point in P^. This implies 
n < 4. We fix n = 4 and fi = Ci for i = 1,2,3,4. Up to permuting and 
rescaling columns, our assumption implies that the configuration A equals 



A^ 



For this camera configuration, the multiview ideal Ja is indeed a toric ideal: 



"0 10 0" 




"10 0" 




"10 0" 




"10 0" 


10 


, ^2 = 


10 


, ^3 = 


10 


, ^4 = 


10 


1. 




1 




1 
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Proposition 4.1. The ideal J a is obtained by eliminating the diagonal un- 
knowns wi, W2, W3 and W4 from the ideal o/2 x 2-minors of the 4 x A-matrix 



(13) 



This toric ideal is minimally generated by six quadrics and four cubics: 
J A = {yiyi-XiZi,y3Xi-X:iyi,y2Xi-X2Zi, ziy^i-xiz^, Z2X:i-X2Z:i, Ziy2-yiZ2, 

y2Z3yi - Z2y3Z4, yiZ3X4 - Z1X3Z4, X1Z2X4 - ZiX2y4, Xiy2X-i - yiX2y3) 



(wi 


X2 


xz 


2:4 \ 


Xl 


W2 


ys 


yi 


yi 


y2 


W3 


Z4 


\zi 


Z2 


Z3 


W4/ 



We extend Aj to a 4 x 4-inatrix Bi as in Section 2 by adding the 



Proof: 

row hi = ef . The -Bj's are then ah permutation matrices, and the matrix in 
([7|) equals the matrix in (13). The ideal J is generated by the 2x2 minors 
of that matrix of unknowns. The multiview ideal is J a = J^ H K[x,y,z]. 
We find the listed binomial generators by performing the elimination with 
a computer algebra package such as Macaulay2. Toric ideals are precisely 
those prime ideals generated by binomials and hence Ja is a toric ideal. D 

Remark 4.2. The normalized coordinate system in multiview geometry pro- 
posed by Heyden and Astrom |12] is different from ours and does not lead 
to toric varieties. Indeed, if one uses the camera matrices in |12[ §2.3], then 
J A is also generated by six quadrics and four cubics, but seven of the ten 
generators are not binomials. One of the cubic generators has six terms. D 

In commutative algebra, it is customary to represent toric ideals by integer 
matrices. Given A G N^^"? with columns ai, . . . , Oq, the toric ideal of A is 

/_4 := (r-i'^ : Au = Av,u,v£n'') C K[t] := K[ti, . . . ,tg], 

where t" represents the monomial i^^ig^ ■ ■ ■tq'' . If ^' is the submatrix of A 
obtained by deleting the columns indexed by ji, . . . ,js for some s < q, then 
the toric ideal /^' equals the elimination ideal I_4 n K[tj : j {ji, . . . ,js}]; 
see [201 Prop. 4.13 (a)]. The integer matrix A for our toric multiview ideal 
Ja in Proposition |4.1| is the following Cayley matrix of format 8 x 12: 

"4^ AT AT aT- 
^1 ^2 ^3 ^4 

10 

^=0100 

10 

^0001 

where 1 = [111] and = [0 0]. This matrix A is obtained from the 
following 8 X 16 matrix by deleting columns 1, 6, 11 and 16: 



(14) 



h 


h 


h 


h 


1 














1 














1 














1 
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Figure 4. Initial monomial ideals of the toric multiview 
variety correspond to mixed subdivisions of the truncated 
tetrahedron P. These have 4 cubes and 12 triangular prisms. 



The vectors 1 and now have length four, I4 is the 4x4 identity matrix 



and we assume that the columns of ( 14 ) are indexed by 



Wl,Xi,yi,Zi,X2,W2,y2, Z2, X3, ^3, W3, Z3, X4, 7/4, Z4, W4. 



The matrix (14) represents the direct product of two tetrahedra, and its 



toric ideal is known (by [2^ Prop. 5.4]) to be generated by the 2x2 minors 
of ( 13 ). Its elimination ideal in the ring K[x, y, z] is /yi, and hence Ja = Ia- 



The matrix A has rank 7 and its columns determine a 6-dimensional poly- 
tope conv(^) with 12 vertices. The normalized volume of conv(^) equals 
16, and this is the degree of the 6-dimensional projective toric variety in P^^ 
defined by J a- In our context, we don't care for the 6-dimensional variety in 
P^-^ but we are interested in the threefold in P^xP^xP^xP^ cut out by Ja- 
To study this combinatorially, we apply the Cayley trick. This means we 
replace the 6-dimensional polytope conv(^) by the 3-dimensional polytope 

P = conv(74f ) + conv(74^) + conv{Aj) + 001^(^1). 

This is the Minkowski sum of the four triangles that form the facets of the 
standard tetrahedron. Equivalently, P is the scaled tetrahedron 4A3 with 
its vertices sliced off. Triangulations of A correspond to mixed subdivisions 
of P. Each 6-simplex in A becomes a cube or a triangular prism in P. Each 



mixed subdivision has four cubes 



and twelve triangular prisms 



x 



Such a mixed subdivision of P is shown in Figure [4| Note the 



similarities and differences relative to the complex V{M4) in Example 3.4 



We worked out a complete classification of all mixed subdivisions of P: 

Theorem 4.3. The truncated tetrahedron P has 1068 mixed subdivisions, 
one for each triangulation of the Cayley polytope conv(^). Precisely 1002 of 
the 1068 triangulations are regular. The regular triangulations form 48 sym- 
metry classes, and the non-regular triangulations form 7 symmetry classes. 
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A 



A^n A 



A ^A ^A A 



A A D 



A ^D ^D A 



A 
Figure 5. The dual graph of the mixed subdivision given by Yi. 



We offer a brief discussion of this result and how it was obtained. Using 
the software Gfan [T3] . we found that Ij[ has 1002 distinct monomial initial 
ideals. These ideals fall into 48 symmetry classes under the natural action 
of (S's)^ XI 5*4 on K[x, y, z] where the i-th copy of S3 permutes the variables 
Xi,yi,Zi, and S4 permutes the labels of the cameras. The matrix A being 
unimodular, each initial ideal of I^ is squarefree and each triangulation 
of A is unimodular. To calculate all non-regular triangulations, we used 
the bijection between triangulations and ^-graded monomial ideals in [201 
Lemma 10.14]. Namely, we ran a second computation using the software 
package CaTS IH] that lists all ^-graded monomials ideals, and we found 
their number to be 1068, and hence A has 66 non-regular triangulations. 

The 48 distinct initial monomial ideals of the toric multiview ideal Ja can 
be distinguished by various invariants. First, their numbers of generators 
range from 12 to 15. There is precisely one initial ideal with 12 generators: 

Yl = {yiZ2,Ziy3,XiZ4„Z2X3,y2X4^,X3y4, 

Xiy2X3, Ziy2X3, X1Z2X4,, Z1X3Z4, Z2y3X4, ^2^32^4 )• 

At the other extreme, there are two classes of initial ideals with 15 genera- 
tors. These are the only classes having quartic generators, as all ideals with 
< 14 generators require only quadrics and cubics. A representative is 

5^2 = {ziy2,XlZ3,XiZi,X2Z3,y2X4„y3Xi, yiZ2X3y4, 

Xiy2X3, XiZ2X3,XiZ2X4, X4Z2yi, yiZ3X4, yi 2:32/4, y2X3yi, 2/22:32/4 )• 

All non-regular ^-graded monomial ideal have 14 generators. One of them is 

Y3 = {ziy2,Ziy3,XiZ4,X2Z3,X2Z4„y3X4„ Xiy2Z3,yiX2y3, 

xiy2X4, X1Z2X4, xizsXi, 2/1Z3X4, y2Z3Xi, 2/22:32/4 )• 
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Al 


Al 


^^1 




1 













1 













1 





A more refined combinatorial invariant of the 55 types is the dual graph of 
the mixed subdivision of P. The 16 vertices of this graph are labeled with 
squares and triangles to denote cubes and triangular prisms respectively, 
and edges represent common facets. The graph for Yi is shown in Figure [5] 

For complete information on the classification in Theorem 4.3 see the 
website www.math.washington.edu/~aholtc/HilbertScheme. 

That website also contains the same information for the toric multiview 
variey in the easier case of n = 3 cameras. Taking Ai,A2 and A-^ as camera 
matrices, the corresponding Cayley matrix has format 7x9 and rank 6: 

00010010 

10 10 

10 10 

A = ^ y JJ = 001001001 

1110 
1110 
111 

This is the transpose of the matrix j4n23} in (Pj) when evaluated at xi = yi = 
• • • = Z3 = 1. The corresponding 6-dimensional Cayley polytope conv(^) 
has 9 vertices and normalized volume 7, and the toric multiview ideal equals 

(15) Ja = {ziy3-xiZ3,Z2X3-X2Z3,ziy2-yiZ2,xiy2X3-yiX2y3)- 
We note that the quadrics cut out Va plus an extra component P^ x P^ x P-*^: 

(16) {ziy3 - X1Z3, Z2X3 - X2Z3, ziy2 - yiZ2) = Ja n (zi, ^2, 23) 

This equation is precisely |12| Theorem 5.6] but written in toric coordinates. 

The toric ideal Ja has precisely 20 initial monomial ideals, in three sym- 
metry classes, one for each mixed subdivision of the 3-dimensional polytope 

P = conv{Ai) +conv{A2) +conv{A'^). 

Thus P is the Minkowski sum of three of the four triangular facets of the 
regular tetrahedron. Each mixed subdivision of P uses one cube ^ 
and six triangular prisms P^ x 



A picture of one of them is seen in Figure [IJ 



Remark 4.4. Our toric study in this section is universal in the sense that 
every multiview variety Va for n < 4 cameras in linearly general position in 
P'^ is isomorphic to the toric multiview variety under a change of coordinates 
in (P^)". This fact can be proved using the coordinate systems for the 
Grassmannian Gr(4, 3n) furnished by the construction in [211 §4]. Here is 
how it works for n = 4. The coordinate change via PGL(3, K) gives 



(17) [AJ Al Al Al] 



* * * 
* * * 



* * * 
* * * 
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where the 3 x 3-matrices indicated by the stars in the four blocks are invert- 
ible. Now, the 4 x 12-niatrix ( |17[ ) gives a support set S that satisfies the con- 
ditions in |2H Proposition 3.1]. The corresponding Zariski open set U^: of the 
Grassmannian Gr(4, 12) is non-empty. In fact, by [21^ Remark 4.9(a)], the 
set Us represents configurations whose cameras /i,/2,/3,/4 are not copla- 
nar. Now, Theorem 4.6 in |21| completes our proof because (the universal 
Grobner basis of) the ideal Ja depends only on the point in U^ C Gr(4, 12) 
represented by (17) and not on the specific camera matrices Ai, . . . , A4. D 



5. Degeneration of Collinear Cameras 

In this section we consider a family of collinear camera positions. The 
degeneration of the associated multiview variety will play a key role in prov- 
ing our main results in Section 6, but they may be of independent interest. 
Collinear cameras have been studied in computer vision, for example in [11]. 

Let e be a parameter and fix the configuration A(e) := {Ai, . . . , An) where 



A,, 



1 
10 
1 



The focal point of camera i is /, = (— 1 : 1 : 1 : e"~*) and hence the n 
cameras given by A{e) are collinear in P^. Note that these camera matrices 
stand in sharp contrast to those for which A is generic which was the focus 
of Sections 2 and 3. They also differ from the toric situation in Section 4. 

We consider the multiview ideal JA{e) ™ the polynomial ring K{e)[x, y, z], 
where K{e) is the field of rational functions in e with coefficients in K. Then 



Ja{e) h^s the Hilbert function (|11|), by Theorem |3.7| Let Qn be the set of 



polynomials in K{e)[x,y, z] consisting of the (2) quadratic polynomials 

(18) Xiyj - Xjyi for 1 <i < j <n 

and the 3(0) cubic polynomials below for all choices ofl<i<j<A;< 



n: 



(19) 



-n—k 



(e"- 



)ziXjXk + (e"" 
)ziyjyk + (e"" 
e"-~'')yiZjXk + {e"-~^ - e"-~^)ziyjXk + (e"" 



'■)xiZjXk + (e" ^ 
'')yiZjyk + (e""^ 



-.n—k 
^n—k 
n—k\ 



-n-r 



XiX j Z}^ 



e" ^)yiyjZk 

■e'^~^)yiXjZk 



first two terms in (19): 



Let Ln be the ideal generated by ( 18 ) and the following binomials from the 



L. 



\ Jb' 



Vj - Xjyi : l<i<j<n) + 



ViZjyk ■ 

yiZjXj^ 



ZiXjXf^, \ 

Ziyjyk, ■ l<i<j<k<n 

ZiyjXk I 



Let Nn be the ideal generated by the leading monomials in (18) and (19): 
Nn ■■= {xiyj : l<i<j<n) + {xiZjXk, yiZjyk, yiZjXk : l<i<j<k<n). 
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The main result in this section is the following construction of a two-step 
flat degeneration JA(e) ~^ Ln^ Nn- This gives an explicit realization of ([2|). 
We note that V^(£) can be seen as a variant of the Mustafin varieties in |2j . 

Theorem 5.1. The three ideals JAie); ^n cind Nn satisfy the following: 

(a) The multiview ideal JaIe) ^-^ generated by the set Qn- 

(b) The binomial ideal Ln equals the special fiber of Ja{£) for e = 0. 

(c) The monomial ideal Nn is the initial ideal of Ln, in the Grobner basis 
sense, with respect to the lexicographic term order with x >~ y >~ z. 

The rest of this section is devoted to explaining and proving these results. 
Let us begin by showing that Qn is a subset of J Ale) ■ The determinant of 



A{e){ij} 



A 
A 



Pi 
3 Pj 



equals (e" ■'— e" ^){xiyj — Xjyi). Hence J^(£) contains (18), by the argument 



in Lemma |2.2[ Similarly, for any l<i<j<k<n, consider the 9x7 matrix 



Ai'"'){ijk} 





1 








Xi 













1 





Vi 








^n^i 








1 


Zi 










1 











Xj 










1 








Vj 





e"-J 








1 





^j 







1 














Xk 







1 











Vk 


^n—k 








1 








Zk 



The three cubics (19), in this order and up to sign, are the determinants 



of the 7x7 submatrices of A{e)^ijkj obtained by deleting the rows corre- 
sponding to yj and yk, the rows corresponding to Xj and x^, and the rows 
corresponding to Xi and yk respectively. We conclude that Qn lies in JA{e) ■ 



We next discuss part (b) of Theorem 5.1, Every rational function c(e) G 
K{e) has a unique expansion as a Laurent series cie""^ + C2£^^ + • • • where 
Ci G K and ai < 02 < ■ ■ ■ are integers. The function val : K{e) — t- Z given by 
c(e) 1-^ ai is then a valuation on K{e), and -fC|e] = {c G K{e) : val(c) > 0} 
is its valuation ring. The unique maximal ideal in i^|e] is m = (c G K{£) : 
val(c) > 0). The residue field -fC|e]/m is isomorphic to K, so there is a 
natural map i^|e] — )• -fC that represents the evaluation at e = 0. The special 
fiber of an ideal / C K{e)[x, y, z] is the image of / n -ff |e][x, y, z] under the 
induced map /f |e][[a;, y, z] -^ K[x,y,z]. The special fiber is denoted in(/). 
It can be computed from / by a variant of Grobner bases (cf. [161 §2.4]). 

What we are claiming in Theorem 5.1 (b) is the following identify 



in(-^A(e)) = Ln in K[x,y,z] 
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It is easy to see that the left hand side contains the right hand side: indeed, 



by multiplying the trinomials in ( 19 ) by e " and then evaluating at e = 0, 
we obtain the binomial cubics among the generators of -L„- 

Finally, what is claimed in Theorem |5.1| (c) is the following identity 

in^(L„) = Nn in K[x,y,z]. 

Here, in_^(L„) is the lexicographic initial ideal of Ln, in the usual Grobner 
basis sense. Again, the left hand side contains the right hand side because 
the initial monomials of the binomial generators of Ln generate N^. 

Note that A^^^ is distinct from the generic initial ideal M„. Even though 
Mn played a prominent role in Sections 2 and 3, the ideal Nn will be more 
useful in Section 6. The reason is that M„ is the most singular point on the 
Hilbert scheme Tin while, as we shall see, Nn is a smooth point on Tin- 

In summary, what we have shown thus far is the following inclusion: 

(20) Nn C in^(in(J^(,))) 

We seek to show that equality holds. Our proof rests on the following lemma. 



Lemma 5.2. The monomial ideal Nn has the Z"' -graded Hilbert function ( 11 ) 



Proof: Let u = {ui, . . . , u„) G N", and let ^u be the set of all monomials 
of multidegree u in K[x, y, z] which are not in Nn- We need to show that 

1^ I _ /ui-l \- Un + 3\ _ -^ /Ui + 2 

' "' " V 3 y ^V 3 

It can be seen from the generators of Nn that the monomials in ^u are 
of the form z"'y^x'^z'^ for a, b,c,d€ N^ such that u = a-\-b-\-c + d and 

a = (ai,...,aj,0,...,0) 

6 = (0,...,0,6„...,6j,0,...,0) 

c = (0, ...,0,Cj,...,Cfc,0, ...,0) 

d = (0, ...,0,(ifc,...,d„) 

for some triple i,j, k with l<i<j<k<n. 

We count the monomials in 55^ using a combinatorial "stars and bars" 
argument. Each monomial can be formed in the following way. Suppose 
there are ui + ■ ■ ■ + Un + 3 blank spaces laid left to right. Fill exactly three 
spaces with bars. This leaves ui + ■ ■ ■ + Un open blanks to fill in, which is 
the total degree of a monomial in QS^. The three bars separate the blanks 
into four compartments, some possibly empty. From these compartments 
we greedily form a, b, c, and d to make z°'y^x'^z'^ as described below. 

In what follows, * is used as a placeholder symbol. Fill the first ui blanks 
with the symbol *i , the next U2 blanks with *2 , and continue to fill up until 
the last Un blanks are filled with *.„. Now we pass once more through these 
symbols and replace each -k^ with either Xj, yi, or Zi such that all variables 
in the first compartment are z's, those in the second are y's, then x's and 
in the fourth compartment z's. Removing the bars gives z°"]^x'^z'^ in ^B^. 
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There are I " ) ways of choosing the three bars. The 

monomials in 53^ are overcounted only when i = j = k if Zi appears in 
both the first and fourth compartments. Indeed, in such cases if we require 
aj = 0, the monomial is uniquely represented, so we are overcounting by the 

choices when ai ^ 0. D 



3 

We are now prepared to derive the main result of this section. 



Proof of Theorem \5.1\ - Lemma 5.2 and Theorem 3.7 tell us that Nn and 
JA(e) have the same Z"-graded Hilbert function (11). We also know from 
|16l §2.4] that in(J^(£-)) has the same Hilbert function, just as passing to an 
initial monomial ideal for a term order preserves Hilbert function. Hence the 



equality A^^^ C in_^(in(J^(£-))) holds in (20). This proves parts (b) and (c) 
We have shown that Qn is a Grobner basis for the homogeneous ideal JA(e) 
in the valuative sense of [HI §2.4]. This implies that Gn generates Jyi(e)- ^ 

Remark 5.3. The polyhedral subcomplexes of (A2)" defined by the bino- 
mial ideal L„ and the monomial ideal Nn are combinatorially interesting. 
For instance, L„ has prime decomposition Is n /4 n • • • n /„ n In+i, where 

It := {xi,yi : i = t,t + 1, . . . ,n) + 

{ XiHj - Xjyi : 1 < i < j < t) + 

{ XiZj - XjZi, yiZj - yjZi : l<i<j<t-l). 

The monomial ideal Nn is the intersection of in^ {It) for t = 3,...,n-|-l. D 

6. The Hilbert Scheme 
We define Tin to be the multigraded Hilbert scheme which parametrizes 



all ^"-homogeneous ideals in K[x,y,z] with the Hilbert function in ( 11 ). Ac- 
cording to the general construction given in [10], Tin is a projective scheme. 
The ideals Ja and in_<(J^) for n distinct camera positions, as well as the 
combinatorial ideals Mn,Ln and A^^ all correspond to closed points on Tin- 
Our Hilbert scheme Tin is closely related to the Hilbert scheme H/j^^n which 
was studied in |3J. We already utilized results from that paper in our proof 



of Theorem 2.1 , Note that H^^n parametrizes degenerations of the diagonal 
P^ in (P^)" while Tin parametrizes blown-up images of that P^ in (P^)". 

Let G = PGL(3,K) and B C G the Borel subgroup of lower-triangular 
3x3 matrices modulo scaling. The group G" acts on K[x,y,z] and this 
induces an action on the Hilbert scheme Tin- Our results concerning the 
ideal M„ in Section 3 imply the following corollary, which summarizes the 
statements analogous to Theorem 2.1 and Corollaries 2.4 and 2.6 in [3J. 

Corollary 6.1. The multigraded Hilbert scheme Tin is connected. The point 
representing the generic initial ideal Mn lies on each irreducible component 
of Tin- All ideals that lie on Tin o,re radical and Cohen-Macaulay. 
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In particular, every monomial ideal in Tin is squarefree and can hence be 
identified with its variety in (P^)", or, equivalently, with a subcomplex in 
the product of triangles (A2)". One of the first questions one asks about 
any multigraded Hilbert scheme, including Tin, is to list its monomial ideals. 

This task is easy for the first case, n = 2. The Hilbert scheme ^2 
parametrizes Z^-homogeneous ideals in K[x,y,z] having Hilbert function 



/i2 :N2^N, (ui,U2) 



1-^ 



^1 + M2 + 3 
3 



ui + 2 
3 



U2 + 2 
3 



There are exactly nine monomial ideals on 712, namely 

(3:1x2), (3:12/2), (2:1^2), (yiX2), {yiy2), {yiZ2), (^12:2), {ziy2), {ziZ2). 

In fact, the ideals on ?^2 are precisely the principal ideals generated by 
bilinear forms, and T-L2 is isomorphic to an 8-dimensional projective space 

"^2 = {{coXiX2 + cixiy2 H h C8Z1Z2) : (cq : ci : • • • : cs) G P^}. 

The principal ideals Ja which actually arise from two cameras form a 
cubic hypersurface in this ?^2 — P^. To see this, we write Aj for the j-th 
row of the i-th camera matrix and lA-j^ A-^^ A-i^ A-'/] for the 4 x 4-determinant 
formed by four such row vectors. The bilinear form can be written as 



xf'Fxi = [X2 1/2 Z2] 



where F is the fundamental matrix |llj . In terms of the camera matrices 



Co 


C3 


ce" 




Xi 


Cl 


C4 


C7 




Vi 


C2 


C5 


C8_ 




_z\_ 



(21) 



F 



{A\A\AlA^^ 
\A\A\A\A% 



\A\A\AlA^^ 
\A\A\A\A^^ 



{A\A\AlA^^ 
\A\A\A\A^^ 



f/l2 43 /ll42l _\ A\ A-i A\ Al^ \a1a2a1a2 

[^1^1^2^2J l^l^l^2^2J l^l^l^2^2 



This matrix has rank < 2, and every 3 x 3-matrix of rank < 2 can be written 
in this form for suitable camera matrices Ai and A2 of size 3x4. 

The formula in (21) defines a map (^1,^2) i— ^ F from pairs of camera 



matrices with distinct focal points into the Hilbert scheme 'H2- The closure 
of its image is a compactification of the space of camera positions. We now 
precisely define the corresponding map for arbitrary n >2. The construction 
is inspired by the construction due to Thaddeus discussed in O Example 7] . 

Let Gr(4, 3n) denote the Grassmannian of 4-dimensional linear subspaces 
of K^^. The n-dimensional algebraic torus (K*)^ acts on this Grassman- 
nian by scaling the coordinates on K^^, where the ith factor K* scales the 
coordinates indexed by 3i — 2,3i — l and 3i. Thus, if we represent each point 
in Gr(4, 3n) as the row space of a (4 x 3n)-matrix ^Aj A^ ■ ■ ■ A^ , then 
A = (Ai,...,A„) e {K*Y sends this matrix to [\iA\ X2A2 ■■■ XnA^]. 
The multiview ideal Ja is invariant under this action by (ET*)". In symbols, 
J\oA = Ja- In the next lemma, GIT stands for geometric invariant theory. 
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Lemma 6.2. The assignment A >-^ Ja defines an injective rational map 7 
from a GIT quotient Gr(4, 3n)//(i^*)" to the multigraded Hilhert scheme Tin- 
Proof: For the proof it suffices to check that Ja 7^ Ja' whenever A and A' 
are generic camera configurations that are not in the same (iT*) "-orbit. D 

We call 7 the camera map. Since we need 7 only as a rational map, the 
choice of linearization does not matter when we form the GIT quotient. The 
closure of its image in Tin is well-defined and independent of that choice of 
linearization. We define the compactified camera space, for n cameras, to be 



r„ := 7(Gr(4,3n)//(i^*)") C Tin- 

The projective variety Tn is a natural compactification of the parameter 
space studied by Heyden in [13]. Since the torus (K*)^ acts on Gr(4, 3n) 



with a one-dimensional stabilizer, Lemma 6.2 implies that the compactified 
space of n cameras has the dimension we expect from [13] , namely, 

dim(r„) = dim(Gr(4, 3n)) - (n - 1) = 4(3n - 4) - (n - 1) = lln - 15. 

We regard the following theorem as the main result in this paper. 

Theorem 6.3. For n > 3, the compactified camera space Tn appears as a 
distinguished irreducible component in the multigraded Hilhert scheme Tin- 

Note that the same statement if false for n = 2: r2 is not a component of 



7^3 ~ P°. It is the hypersurface consisting of the fundamental matrices (21 ). 

Proof: By definition, the compactified camera space r„ is a closed sub- 
scheme of Tin- The discussion above shows that the dimension of any irre- 
ducible component of Tin that contains r„ is no smaller than lln — 15. We 
shall now prove the same lln — 15 as an upper bound for the dimension. 
This is done by exhibiting a point in r„ whose tangent space in the Hilbert 
scheme Tin has dimension lln — 15. This will imply the assertion. 

For any ideal I G Tin, the tangent space to the Hilbert scheme Tin at / is 
the space of K[x, y, z]-module homomorphisms I — )• K[x, y, z]/I of degree 0. 
In symbols, this space is }iom.(I,K[x,y,z]/I)o- The X-dimension of the 
tangent space provides an upper bound for the dimension of any component 
on which / lies. It remains to specifically identify a point on F^ that is 
smooth on Tin, an ideal which has tangent space dimension exactly lln— 15. 

It turns out that the monomial ideal A^„ described in the previous section 



has this desired property. Lemmas 6.4 and 6.5 below give the details. D 



Lemma 6.4. The ideals Ln and Nn from the previous section lie in F„. 
Proof: The image of 7 in Tin consists of all multiview ideals J a, where A 



runs over configurations of n distinct cameras, by Theorem 3.7 Let A{£) 
denote the collinear configuration in Section 5, and consider any specializa- 
tion of e to a non-zero scalar in K. The resulting ideal JA(e) is a i^-valued 
point of r„, for any e G ir\{0}. The special fiber Ja(o) = Ln is in the Zariski 
closure of these points, because, locally, any regular function vanishing on 
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the coordinates of JaIe) fo^ all e 7^ will vanish for e = 0. We conclude 
that Ln is a X-valued point in the projective variety r„. Likewise, since 
Nn = in^(L„) is an initial monomial ideal of L„, it also lies on r„. D 

Lemma 6.5. The tangent space of the multigraded Hilhert scheme Tin at 
the point represented by the monomial ideal Nn has dimension lln — 15. 

Proof: The tangent space at Nn equals Houi^Nn, K[x,y,z]/Nn)o. We 
shall present a basis for this space that is broken into three distinct classes: 
those homomorphisms that act nontrivially only on the quadratic generators, 
those that act nontrivially only on the cubics, and those with a mix of both. 
Each K[x,y, z]-module homomorphism (p : Nn -^ K[x,y,z]/Nn below is 
described by its action on the minimal generators of Nn- Any generator not 
explicitly mentioned is mapped to under ip. One checks that each is in fact 
a well-defined K[x, y, z]-module homomorphism from Nn to K[x, y, z]/Nn- 

Class I: For each 1 < i < n, we define the following maps 

• tti : XiVk ^ Viyk for all i < k < n, 

• (3i : Xiyi+i i-> Xi+iyi. 

For each 1 < A; < n, we define the following map 

• Ik '■ Xiyk I— ?• XiXk for all 1 < i < k. 
We define two specific homomorphisms 

• Si : xiy2 i-> yiZ2, 

• ^2 ■ Xn-lVn l-> Zn-lXn- 

Class II: For each 1 < j < n, we define the following maps. Each 
homomorphism is defined on every pair (i, k) such that l<i<j<k<n. 

• Pj '■ XiZjXji I— >■ XiXjXj^ and yiZjX}^ 1— )■ yiXjX^;, 

• Cj : XiZjX]^ I— >■ XiXjZf; and yiZjXi^ 1— >■ yiXjZ}^, 

• Tj : XiZjXf; I— >■ XiZjZj^ and yiZjX}^ 1— >■ yiZjZ^;, 

• Uj : yiZjXk ^ ViyjXk and yiZjyk ^ ytyjyk, 

• fij : ViZjXk ^ ZiVjXk and yiz^yu ^ Ziyjyk, 

• TTj : yiZjXk ^ ZiZjXk and yiZjyt ^ ZiZjyk- 
Class III: For each 1 < i < n, we define the map 

• ej : Xiyk 1— )■ Ziyk and XiZjXk 1— 5- ZiZjXk for i < k < n and i < j < k. 
For each 1 < A; < n, we define the map 

• Cfc : Xiyk ^ XiZk and yiZjyk ^ yiZjZk iov 1 < i < k and i < j < k. 

All these maps are linearly independent over the field K. There are n — 1 
maps each of type a^, /3i, 7^, e^, and Cfc; for a total of 5 (n — 1) different 
homomorphisms. Each subclass of maps in class II has n — 2 members, 
adding 6(n — 2) more homomorphisms. Finally adding Si and 82, we arrive 
at the total count of 5(n — 1) + 6(n — 2) + 2 = lln — 15 homomorphisms. 

We claim that any K[x,y, z] -module homomorphism Nn — > K[x,y, z]/Nn 
can be recognized as a i^-linear combination of those from the three classes 
described above. To prove this, suppose that ip : Nn — )■ K[x,y,z]/Nn is a 
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module homomorphisni. For 1 < i < A: < n, we can write (p{xiyk) as a linear 
combination of monomials of multidegree e^ + e^ which are not in Nn- By 
subtracting appropriate multiples of a,, e^, 7^, and Cfcj we can assume that 

'fixiVk) = a yiXk + b yiZk + c ZjXfc + d ZiZk 

for some scalars a, b,c,d£ K. We show that this can be written as a linear 
combination of the maps described above by considering a few cases. 

In the first case we assume i + 1 < k. We use K[x, y, zj-linearity to infer 

(p{xiyi+iyk) = ayiVi+iXk + byiyi+iZk + cziyi+iXk + dziyi+iZk = yk-i^ixiyi+i). 

Specifically, yk divides the middle polynomial. But none of the four mono- 
mials are zero in the quotient K[x,y, z]/Nn. Hence, = a = b = c = d. 

For the subsequent cases we assume k = i + 1. This allows us to further 
assume that a = 0, since we can subtract off a I3i{xiyi+i). Now suppose that 
we have strict inequality k < n. As before, the K[x, y, z]-linearity of cp gives 

^{xiykyn) = dziZkyn = yk^{xiyn). 
Specifically, yk divides the middle term. Hence, d = 0. Similarly, c = 0: 

(fiXiykZkXn) = CZiXkZkXn = Vk '^{XiZkXn)- 

Suppose we further have the strict inequality 1 < i. Then necessarily 6 = 0: 

ip{yiZiXiyk) = byiZiyiZk = Xiip{yiZiyk). 

However, if i = 1 and k = 2, we have that <f{xiy2) = b5i{xiy2)- 

The only case that remains is k = n and i = n — 1. Here, we can also 
assume that c = by subtracting c52{xn-iyn)- We will show that d = = 6 
by once more appealing to the fact that c/? is a module homomorphisni: 

Lp{xiXn-^iyn) = dxiZn-lZn = X„_l Lp{xiyn), 

which gives d = 0. This subsequently implies the desired 6 = 0, because 

ifiyiXiZiyn) = byiyiZiZn = xnp{yiZiyn). 

This has finally put us in a position where we can assume that (p{xiyk) = 
for all 1 < i < fc < n. To finish the proof that 99 is a linear combination 
of the lln — 15 classes described above, we need to examine what happens 
with the cubics. Suppose l<i<j<k<n, and consider ip{yiZjXk)- This 
can be written as a linear sum of the 17 standard monomials of multidegree 
Cj + Cj + Ck which are not in Nn- Explicitly, these standard monomials are: 

XiXjXki XiXjZk, XiZjZk, ViXjXki yiXjZk 

ViVjXk, yiViVk, ViVjZk, ViZjZk, 
ZiXjXki ZiXjZkt ZiyjXki Zj^yjyk, 

ZiyjZki ZiZjXki ZiZjyki ZiZjZk- 

By subtracting off multiples of the maps pj , aj , Tj , Vj , /ij , and ttj , we can 
assume that this is a sum of the 11 monomials remaining after removing 
yiXjXk, yiXjZk, yiZjZk, yiyjXk, ZiyjXk, and ZiZjXk- However, now note that 

ip{xiyiZjXk) = Xi ifiyiZjXk) = yi ip{xiZjXk). 
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This means that for every one of the 11 monomials m appearing in the sum, 
either xjtti = or y, divides m. Similarly, 

(piyiZjXkVk) = ykfiviZjXk) = XkfiviZjyk), 

and so either ykin = or Xk divides m. Taking these both into consideration 
actually kills every one of the 11 possible standard monomials (we spare the 
reader the explicit check), and hence we can assume that ip{yiZjXk) = 0. 
Now consider what happens with (p{xiZjXk)- Indeed, 

= Xi ip{yiZjXk) = ip{xiyiZjXk) = yi ip{xiZjXk). 

So for every one of the 17 standard monomials m which possibly appears in 
the support of ip{xiZjXk) we must have that yim = in K[x, y, z\/Nn- This 
actually leaves us with only two possible such standard monomials - namely 
ZiZjXk and ZiZjyk- We write ip{xiZjXk) = aziZjXk + bzjZjyk- 

The fact that we assume ip{xiyk) = implies a = = 6. This is because 

= ZjXk ^{xiyk) = ip{xiZjXkyk) = Vk (p{xiZjXk). 

To sum up, we have shown that, under our assumptions, if ip{yiZjXk) = 
holds then it also must be the case that (p{xiZjXk) = 0. We can prove in a 
similar manner that (p{yiZjyk) = 0, and this finishes the proof that if can be 
written as a i^-linear sum of the lln — 15 classes of maps described. D 



We reiterate that Theorem 6.3 fails for n = 2, since ^2 — IP > and r2 is a 



cubic hypersurface cutting through ^2- We offer a short report for n = 3. 

Remark 6.6. The Hilbert scheme Tis contains 13, 824 monomial ideals. 
These come in 16 symmetry classes under the action of (S^)^ xi 53. A detailed 
analysis of these symmetry classes and how we found the 13, 824 ideals ap- 
pears on the website www . math . Washington . edu/~aholtc/HilbertScheme. 
For seven of the symmetry classes, the tangent space dimension is less than 
dim(r3) = 18. From this we infer that Ti^ has components other than T^. 

We note that the number 13, 824 is exactly the number of monomial 
ideals on H^^^ as described in [3J. Moreover, the monomial ideals on H33 
also fall into 16 distinct symmetry classes. We do not yet fully understand 
the relationship between Tin and H^^n suggested by this observation. 

Moreover, it would be desirable to coordinatize the inclusion r3 C Tis 
and to relate it to the equations defining trifocal tensors, as seen in [H [13] . 
It is our intention to investigate this topic in a subsequent publication. 

Our study was restricted to cameras that take 2-dimensional pictures of 
3-dimensional scenes. Yet, residents of flatland might be more interested in 
taking 1-dimensional pictures of 2-dimensional scenes. From a mathemati- 
cal perspective, generalizing to arbitrary dimensions makes sense: given n 
matrices of format r x s we get a map from F^~^ into (P''~^)", and one could 
study the Hilbert scheme parametrizing the resulting varieties. Our focus 
on r = 3 and s = 4 was motivated by the context of computer vision. 
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